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DETAILED ACTION 
Specification 

The disclosure is objected to because of the following informalities: 

Figure 12 is present but not mentioned in the Brief Description of the Drawings. 

On Page 13, the paragraph starting at line 15, 34R and 34L are referred to as 
elbow joints, while in the paragraphs starting at lines 24 and 30, they are referred to as 
knee joints. A reference number may only refer to one part of the invention. 

On Page 14, line 6 reference numbers 41 and 42 are referred to as "neck 
ankles," these are neck joints. 

On Page 1 6, lines 8-10, the sentence is incomplete - the robot grasps the 
position of what? 

Page 32, lines 31- 32 refer to 1 - 3 processes to be determined. On Page 33, 5 
processes are described. 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sl<ill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



Application/Control Number: 1 0/786.31 1 Page 3 

Art Unit: 3609 . 

The factual inquiries set forth in Graham v, John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for deternnining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

Claims 1-4, and 10-11 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over PENG, Jian et al., "An Active Vision System for Mobile Robots," 
Systems, AMN. And Cybernetics, 2000, IEEE International Conference on Nashville, 
TN. USA, October 8- " 1 1 . 2000. Piscataway. NJ. USA. IEEE. U.S. Vol. 2. Pgs. 1472- 
1477 (hereafter referred to as Peng) in view of KUNIYOSHI Y. et al., "A Humanoid 
Vision System for Versatile Interaction," Lecture Notes in Computer Science, Vol. 1811, 
Springer-- Germany, January 2000 (200-01), Pgs. 512-526 (hereafter referred to as 
Kuniyoshi). Peng describes a mobile robot that uses two independently panning 
cameras to mimic the binocular active vision of the human eye. Peng's robot uses color 
cues to find a target and then processes the digital images to track the target. The 
cameras in Peng are equipped with standard lenses rather than applicant's non-central 
projection lenses. Kuniyoshi describes a vision system for a robot that also uses an 
active vision system. The dual cameras in Kuniyoshi are equipped with wide angle 
lenses to provide a wider field of view than the system in Peng. 

Regarding claim 1 , "An automatic work apparatus which detects spatial position 
of targets based on images thereof taken by means of plural observing camera devices 
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that have lenses of non-central projection" reads on Peng's implementation of a robot 
with a head having two color cameras mounted on two independent pan-tilt heads, 
Peng, page 1472, abstract. Peng does not describe using a non-central projection lens 
on these cameras, but Kuniyoshi implements a binocular active vision system with 
foveated wide angle lenses, page 512 last sentence of abstract. This system is further 
described on page 515, in section 3 first 2 ^s. While applicant suggests "such as fisheye 
lenses set therein" - this is only an example and a foveated wide angle lens also has a 
non-central projection. It would have been obvious to one of ordinary skill in the art at 
the time of the invention to incorporate the wide angle lens of Kuniyoshi into Peng's 
system because robotic researchers strive to imitate human functions, and the eye is 
one of the most complicated and precise organs in the human body, Peng 2"^ ^ of Intro, 
page 1472. Detection of the spatial position of targets is mentioned in Peng at page 
1473, 1^*11 of Active Vision System Behaviors. "[A]nd carries out predetemnined tasks," 
reads on the robot head smoothly tracking the moving target and following the target as 
it moves, Peng, page 1472, last sentence of abstract, "a rotation device to. change 
viewing direction of said observing camera devices;" reads on Peng's pan-tilt units that 
support the cameras, page 1473. 1®* H of Hardware configuration, "a target image 
extracting unit to extract said targets in said images taken by means of said plural 
observing camera devices" reads on the fact that Peng detects a target, page 1473, 1^* 
H of Active Vision System Behaviors, "a rotation controller to control said rotation device 
in accordance with a certainty obtained by discriminating the target position information 
in response to target position that is specified by said target image extracting unit for 
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each of said images of said targets" reads on Peng's smooth pursuit module that 
continues to track an object at moderate speed to keep it in focus Page 1473, 1®* ^ of 
Active Vision System Behaviors and discussion of VOR behavior, page 1475, 1^* T|of 
VOR. 

Regarding claim 2, "An automatic work apparatus which specifies spatial 
position of targets based on images thereof taken by means of plural observing camera 
devices that have lenses of non-central projection such as fisheye lenses set therein 
and carries out predetermined tasks comprising: a rotation device to change viewing 
direction of said observing camera devices; a target image extracting unit to extract said 
targets in said images taken by means of said plural observing camera devices" reads 
on the sections of Peng as referenced relative to claim 1 . Further, it would have been 
obvious to one of ordinary skill in the art at the time of the invention to use the wide 
angle lens of Kuniyoshi to extend the range of observation, "a position detemiining unit 
to determine said-spatial positions of said targets by using image positions of said 
targets specified by said target image extracting unit" reads on the description of using 
two neural nets to detect the position of the target, Peng, page 1474, ^ under Figure 2. 
"a rotation controller to control said rotation device so that said rotation device changes 
viewing direction of said observing camera devices which take images, wherein at least 
two areas as a peripheral area and a central area are assigned to said Images, of said 
targets in said central area in case that image positions of said targets specified by said 
target image extracting unit are in said peripheral area" reads on the description of 
smooth pursuit where three zones are defined in the image plane and no head motion is 
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needed when the target is in the dead zone, smooth pursuit is possible while the target 
is in the fovea area, and the saccade module is used to bring the target back into focus 
when the target is outside the fovea area, Peng, 1474 ^ at top of 2"^ column. 

Regarding claim 3 which depends on claim 2. "a[n] transfer equipment to change 
relative distance with said targets so that said predetermined tasks are carried out by 
adjusting relative position against said target on a basis of spatial positions of said 
targets determined by said position determining unit" reads on Peng's VOR unit keeping 
the robot moving toward the target while maintaining both left and right cameras on the 
target, page 1475, U beneath Fig 7, 

Regarding claim 4 which depends on claim 2, "said target image extracting unit 
has a function to extract said targets from said images taken by means of plural 
observing camera devices by specifying only picture elements that correspond to a 
reference color which is assigned for said targets and memorized for identifying said 
targets beforehand" reads on Peng's description of the object -following experiment 
where a color model was used to find a target, page 1476, ^ after VI Experimental 
Results. 

Regarding independent claim 10, "An automatic work control program 
apparatus that, in order to control an automatic work apparatus that comprises plural 
observing camera devices that have lenses of non-central projection such as fisheye 
lenses, to change directions of said observing camera device and to carry out 
predetermined tasks for said target on a basis of information obtained by images taken 
by said plural observing cameras systems" descrit^es the system as described in Claims 



Application/Control Number: 1 0/786,31 1 Page 7 

Art Unit: 3609 

1 and 2 and is rejected based on the same sections of the references that are combined 
for the same reasons, "has a computer functioning for a purpose of means of target 
image extracting unit to extract said targets in said images taken by means of said 
observing camera devices" reads on the computers controlling the robot (Peng page 
1473, 2"^ H in 1^' column) actions in detecting targets as described on Peng, page 1473 
1^* under Active Vision System Behaviors, "a computer functioning for a purpose of 
means of rotational control of the rotational drive unit to control said rotational drive unit 
in accordance with a certainty of positional Information obtained by discriminating said 
target position information regarding said target in response to said target position that 
is specified in each image by said target image extracting unit" reads on the computers 
controlling the head and cameras of the robot in Peng to track the target as moderate 
speed via the smooth pursuit module, page 1473, 3^^ sentence of 1^* II under Active 
Vision System Behaviors. 

Regarding independent claim 11, "An automatic work control program apparatus 
that, in order to control an automatic work apparatus that comprises plural observing 
camera devices that have lenses of non-central projection such as fisheye lenses, to 
change directions of said observing camera device and to carry out predetemnined 
tasks for said target on a basis of information obtained by images taken by said plural 
observing cameras systems, has a computer functioning for a purpose of means of 
target Image extracting unit to extract said targets in said images taken by means of 
said observing camera devices" describes the system as described in Claims 1 and 2 
and is rejected based on the same sections of the references that are combined for the 
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same reasons, "a purpose of means of position evaluation to obtain the target spatial 
position information in response to said target position that is specified in images by 
said target image extracting unit" reads on the description of using two neural nets to 
detect the position of the target, Peng, page 1474, If under Figure 2. "a purpose of 
means of rotation controller to control said rotation device so that said rotation device 
changes viewing direction of said observing camera devices which take images, 
wherein at least two areas as a peripheral area and a central area are assigned to said 
images taken by said observing camera devices, so that said images of said targets are 
taken in said central area by said observing camera devices in case that image 
positions of said targets specified by said target image extracting unit is in said 
peripheral area, reads on the description of smooth pursuit where three zones are 
defined in the image plane and no head motion is needed when the target is in the dead 
zone, smooth pursuit is possible while the target is in the fovea area, and the saccade 
module is used to bring the target back into focus when the target is outside the fovea 
area, Peng, 1474 H at top of 2"^ column. 

Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Jeng and 
Kuniyoshi as applied to claims 1 and 2 above and further in view of Tatsuno, KyoichI, "A 
Beach Ball Volley Playing Robot with a Human", International Journal of the Robotics 
Society of Japan, No. 5, Vol. 18, pp. 105-1 1 1 (2000) (hereafter referred to as Tatsuno). 
Tatsuno describes extending the visual capabilities of a robot to interact with humans by 
batting a ball back and forth. 
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Regarding claim 5, which depends on claim 4, "said extracted target image by 
said target image extracting unit is labeled with a single same ID onto plural picture 
elements which keep a connectivity thereover so that said plural picture elements 
specify said target" reads on the 2"^ ^ of section 5.1 of Tatsuno, where the process of 
identifying the image is detailed and a labeling processor is described. The labeling 
processor evaluates the connectivity of the color of adjacent pixels and uses identical 
labels for the same color. Both Jeng and Kuniyoshi were interested in extending the 
visual acuity of a robot. It would have been obvious to one of ordinary skill in the art at 
the time of the invention to use Tatsuno*s labeling process in identifying the head and 
arms as detailed in Kuniyoshi at page 524, the paragraph under figure 1 1 to improve the 
processing of that data. 

Allowable Subject Matter 

Claims 6-9 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. The following is a statement of reasons for the 
indication of allowable subject matter: 

Claim 6 recites in part to detemnine an identification of the targets specified by 
composures of said picture elements that correspond to a reference color, which is 
assigned for said targets with said targets on a basis of predetermined aspect value 
after computing an aspect ratio of planer expansion of said picture elements. None of 
the cited art teaches or suggests using aspect ratio In this way. 
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Claim 7 recites in part to determine an identification of said targets specified by 
composures of said picture elements that correspond to a reference color which is 
assigned for said targets with said targets on a basis of predetermined filling rate of said 
composures of said picture elements against a rectangular given by a vertical viewing 
angle and a horizontal viewing angle in a viewing angle mapping specified by said 
images taken by means of plural observing camera devices. None of the cited art 
teaches or suggests using the filling rate of a rectangle defined in this way to identify a 
target. 

Claim 8 recites in part, images taken by means of plural observing camera 
devices are sampled with respect to colors and said reference color which is assigned 
for said targets is determined by eliminating a color of which area expands from the 
origin of Cr-Cb space chart after plotting said sampled color onto Cr-Cb space. None of 
the cited references teach or suggest excluding the area that expands from the origin of 
the Cr-Cb space chart in sampling for a color match. 

Claim 9 recites in part, a robot having two observing camera devices set in right 
hand side and left hand side thereof, wherein; said position determining unit comprises 
a distance computing module that detenmines distance to said target therefrom using 
position information of said target image specified by said target image extracting unit, a 
horizontal position computing module that computes horizontal position to said target 
using position information of said target image specified by said target image extracting 
unit and a vertical position computing module that computes vertical position to said 
target using position information of said target image specified by said target image 
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extracting unit, three categories of areas as a left or right peripheral area, an upper or 
lower peripheral area and a central area are set for each image taken by said observing 
camera device. The cited prior art does not teach or suggest having the target image 
extracting unit dividing the target image into those three categories of areas. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure; U.S. Patent No. 6,853,880 for pattern recognition based on 
color,; "Vision for Mobile Robot Navigation; A Survey" for background of the state of the 
art, and "Movie Video Hypermedia Authoring System" for using filling rate to identify a 
character. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Lin B. Olsen whose telephone number is 571-272-9754. 
The examiner can normally be reached on M-F, 7:30am-5:00pm EST, Alternate Fri. off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian T. Pendleton can be reached on 571-272-7527. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR.or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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